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REQUEST  FOR  PROPOSAL 

Test  Development,  Technology,  and  Logistical  Support 
for  the  North  Carolina  Testing  and  Accountability  Program 
2014-15  and  2015-16 


Section  I:  Purpose  of  the  Request  for  Proposal 

The  Division  of  Accountability  Services,  North  Carolina  Department  of  Public  Instruction  (NCDPI)  is  noting  the 
continuation  of  services  provided  by  the  Center  for  Urban  Affairs  and  Community  Services,  North  Carolina  State 
University  (NCSU-TOPS).  In  addition  to  the  Scope  of  Work  specified  in  the  2012-13  and  2013-14  contract, 
NCSU-TOPS  agrees  to  provide  additional  services  as  indicated  in  this  document.  These  services  include  providing 
research  design  and  data  collection,  psychometric  technical  assistance  and  support,  professional  assistance  and 
support,  evaluation  studies,  logistical  solutions  and  coordination  assistance  overlaid  with  the  necessary  technology 
to  ensure  efficiency  in  supporting  and/or  improving  the  (1)  North  Carolina  Testing  Program  and  the  State  Board  of 
Education’s  (SBE)  READY  accountability  program. 

The  major  tasks  and  activities  associated  with  the  2014-15  and  2015-16  scope-of-work  are  defined  in  detail  in  the 
section  of  this  document,  entitled,  “Section  III:  Scope-of-Work.”  However,  as  an  introduction  to  the  organization 
of  the  work  required  to  successfully  implement  the  existing  public  school  testing  and  accountability  programs,  the 
three  major  works  areas  are  identified  and  listed  below: 

1 .  Test  Development  Services  and  Psychometric  Technical  Assistance  and  Support  (60%); 

2.  Technical  Assistance  and  Support,  Programming  Services,  Logistical  Solutions,  Test  Production  and 
Layout,  Warehousing  and  Materials  Delivery,  Online  Testing  Delivery  and  Hosting,  and  Archiving  of 
Documents  (32%);  and 

3.  Research  Design  and  Studies,  Data  Collection  and  Processing,  Statistical  Analyses,  Report  Generation, 
and  Evaluation  Support  (8%); 


Section  II.  Background 

It  is  important  that  as  a  prospective  contractor,  the  leadership  and  the  staff  be  familiar  with  the  current  history  and 
requirements  of  the  North  Carolina  Testing  Program,  including  recent  modifications  to  the  program  and  how  public 
school  educational  testing  programs,  in  general,  are  administered  in  this  state. 

2.1  Background  of  the  North  Carolina  Testing  Program 

The  North  Carolina  Testing  Program  was  initiated  in  response  to  legislation  passed  by  the  North  Carolina  General 
Assembly  in  1977.  Initially,  legislation  addressed  the  implementation  of  the  Annual  Testing  Program,  which 
consisted  of  norm-referenced  testing  at  specified  grades  and  a  minimum  competency  test.  Since  that  time,  the 
program  has  been  expanded  and  currently  includes  state  and  federally  mandated  end-of-grade  curriculum-based 
tests  in  English  language  arts/reading  and  mathematics  at  grades  3-8,  science  at  grades  5  and  8,  and  three  end-of- 
course  tests  (Math  I,  English  II,  and  Biology),  the  Grade  10  College  and  Career  Readiness  Assessment,  the  Grade 
1 1  College  and  Career  Readiness  Assessment,  the  Beginning  of  Grade  3  English  Language  Arts/Reading,  the 
Grade  3  Reading  Portfolio,  the  Read  to  Achieve  Alternate  Test,  and  the  North  Carolina  Final  Exams.  In  addition,  in 
response  to  the  No  Child  Left  Behind  Act  of  2001  and  the  Individuals  with  Disabilities  Act,  North  Carolina  has 
instituted  a  system  of  alternate  assessments  for  students  with  the  most  severe  cognitive  disabilities.  These 
assessments  are  administered  at  the  same  grade  levels  and  in  the  same  content  areas  as  the  general  assessments. 
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With  the  exception  of  the  Grade  3  Portfolio,  the  Read  to  Achieve  Alternate  Test,  and  the  NC  Final  Exams,  all  tests 
administered  in  the  North  Carolina  Testing  Program  are  included  in  the  READY  accountability  program  either  in 
achievement,  growth  and/or  participation  measures.  The  READY  accountability  program  was  implemented  in 
2012-13,  replacing  the  ABCs  of  Public  Education  which  was  in  effect  1996-2012.  With  the  READY 
accountability  program,  schools  are  held  accountable  for  student  achievement  in  the  areas  of  English  language 
arts/reading,  mathematics,  and  science. 

The  current  North  Carolina  Testing  Program  is  administered  also  to  meet  the  reporting  needs  of  the  No  Child  Left 
Behind  Act  (NCLB)  of  2001  (currently  Title  I  Annual  Measurable  Objectives  and  Title  III  requirements  related  to 
students  with  limited  English  Proficiency)  and  the  Individuals  with  Disabilities  Education  Act  (IDEA). 

It  is  an  expectation  of  this  contract  that  the  NCDPI  staff,  with  the  NCSU-TOPS  staff,  will  continue  to  seek  ways  to 
minimize  intrusive  activities  related  to  conducting  studies  or  field  test  activities  necessary  to  validate  the  state  tests. 
The  NCSU-TOPS  operation  along  with  the  North  Carolina  Testing  Program  staff  will  continue  to  explore  ways  to 
implement  a  statewide  testing  program  that  is  effective  and  efficient  and  meets  the  requirement  for  the  agency’s 
vision  and  State  Board  of  Education’s  mandates  for  the  2014-15  and  2015-16  school  years. 


Section  III.  Scope  of  North  Carolina  Testing  Program  for  2014-15  and  2015-16 

3.1  Tests  and  Other  Projects  Required  for  2014-15  and  2015-16 

For  the  2014-15  and  2015-16  school  years  (to  date),  the  North  Carolina  Testing  Program  includes  the  following 
operational  general  assessments,  alternate  assessments,  and  field  tests  or  special  studies. 

1)  English  Language  Arts  (EOG-EOC) 

a.  Grades  3-8  EOG  (multiple-choice) 

b.  English  II  EOC  (multiple-choice/constructed  response) 

c.  Grades  3-8,  10  and  1 1  NCEXTEND1  (performance-based  multiple  choice) 

d.  Beginning  of  Grade  3  (multiple-choice) 

e.  Grade  3  Reading  Portfolio  (multiple-choice) 

f.  Grade  3  Read  to  Achieve  Alternate  (multiple-choice) 

2)  Mathematics  (EOG-EOC) 

a.  Grades  3-8  EOG  (multiple-choice/gridded  response  grades  5-8) 

b.  High  School  Math  I  EOC  (multiple-choice/gridded  response) 

c.  Grades  3-8,  10  and  1 1  NCEXTEND1  (performance-based  multiple-choice) 

3)  Science  (Operational) 

a.  Grades  5  and  8  EOG  (multiple-choice/technology-enhanced) 

b.  Biology  EOC  (multiple-choice/technology-enhanced) 

c.  Grades  5,  8,  10  and  1 1  NCEXTEND1  (performance-based  multiple-choice) 

4)  College  and  Career  Readiness  Alternates  (per  GS  115C-83.1) 

a.  College  and  Career  Readiness  Grade  1 1  -Alternate  (multiple-choice/writing  prompt) 

b.  College  and  Career  Readiness  Grade  10-Alternate  (multiple-choice/writing  prompt) 

5)  North  Carolina  Final  Exams 
a.  English  I  (multiple-choice) 
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b .  English  III  (multiple-choice) 

c.  English  IV  (multiple- Choice) 

d.  Common  Core  Integrated  Math  III  (2014-2615  only)  (multiple- choice) 

e.  Common  Core  Geometry  (20 1 4-2015  only);  (multiple-choice) 
f;  Common  Core  Algebra  II  (2014-2015  only)  (multiple-choice) 

g.  Pre-Calculus  (multiple-choice) 

h.  Discrete  Math  (multiple-choice) 

i.  High  School  Math  II  (multiple-choice) 

j.  High  School  Math  III  (multiple-choice) 

k.  Advanced  Functions  in  Modeling  (multiple-choice) 

L  ,  Earth/Environmental  Science  (multiple-choice) 

m.  Physical  Science  (multiple-choice) 

n.  Chemistry  (multiple-choice) 

o.  Physics  (multiple-choice) 

p.  World  History  (multiple-choice) 

q.  Civics  and  Economics  (multiple-choice) 

r.  American  History  I  (multiple-choice) 

s.  American  History  II  (multiple-choice) 

t.  DS  History  (2014-15  only)  (multiple-choice) 

u.  Science  Grade  4  (multiple-choice) 

v.  Science  Grade  6  (multiple-choice) 

w.  Science  Grade  7  (multiple-choice) 

x.  Social  Studies  Grade  4  (multiple-choice) 

y.  Social  Studies  Grade  5  (multiple-choice) 

z.  Social  Studies  Grade  6  (multiple-choice) 
aa.  Social  Studies  Grade  7  (multiple-choice) 
bb.  Social  Studies  Grade  8  (multiple-choice) 

6)  Special  studies/field  tests  as  approved  by  the  NC  State  Board  of  Education 

a.  Study  of  transition  to  TestNav  online  delivery  system  for  end-of-course  tests  to  ensure  feasibility 
and  comparability 

A  copy  of  the  chart  depicting  the  proposed  North  Carolina  Testing  Program  for  the  2014-15  and  2015-16  school 
years  will  be  posted  on  the  NCDPI  website  at  www.ncpublicschools.org  in  draft  format  at  the  end  of  May. 

In  addition  to  the  operational  assessments  specified  above,  the  Scope  of  Work  includes  the  following  projects: 

1 .  Support  professional  development  resources  in  NCEES  (North  Carolina  Educator  Effectiveness 
System)  for  professional  development  and  NC  FALCON 

2.  Support  the  Honors  Portfolio  submission,  review,  and  report  services 

3.  .  Develop  and  manage  online  distributed  scoring  system  and  conduct  scoring  for  ten  NC  Final  Exam 

Courses  (piloting  in  2013-14  and  operational  in  2014-15) 

4.  Develop  and  maintain  online  testing,  including  the  transition  from  NCTest  to  TestNav 

5.  Develop  classroom  and  benchmark  item  bank,  including  delivery  support  for  the  Schoolnet  component 
of  Home  Base 

6.  Support  SAS  tools,  including  internal  SAS  Server  and  integration/report  host  project  with  SAS 
Business  Intelligence  tools 
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3.2  Non-Public  Schools  Testing  Program. 

In  addition  to  students  in  public  schools,  the  NCDPI  provides  testing  materials  to  non-public  and  federal  schools  on 
an  optional  basis,  as  part  of  the  North  Carolina  Testing  Program.  Non-public  schools  and  federal  schools  may  elect 
to  participate  in  the  North  Carolina  Testing  Program,  on  a  reasonable  fee-basis,  by  registering  with  the  North 
Carolina  Non-Public  Education  Coordinator  at  NCSU-TOPS.  For  the  2014-15  and  2015-16  school  years  all 
ordering,  billing,  and  other  communication  between  the  non-public  schools  and  the  appropriate  contractor  will  be 
for  the  specific  test.  The  North  Carolina  Non-Public  Education  coordinator  will  provide  the  list  of  non-public 
schools  that  have  completed  the  requisite  training  for  test  administration,  and  thus,  authorize  them  to  order  testing 
materials  whether  through  NCSU-TOPS  or  through  an  identified  vendor  which  depends  on  the  assessment 
requirements  and  specifications. 

3.3  Community  Colleges 

In  addition  to  providing  services  to  non-public  and  federal  schools,  the  NCDPI  also  provides  services  to  the  58 
community  colleges  operating  under  the  state  community  college  system.  These  services  provide  community 
colleges  an  opportunity  to  obtain  training  and  technical  support;  to  order  test  materials;  and  scanning,  scoring,  and 
reporting  services  for  tests  currently  required  of  North  Carolina  students  in  order  to  receive  a  North  Carolina  high 
school  diploma  under  the  Adult  High  School  Diploma  Program  including  the  end-of-course  tests  required  to  fulfill 
some  high  school  course  requirements.  The  test  materials  provided  are  to  be  the  same  as  those  used  in  the  North 
Carolina  Testing  Program.  Use  of  these  test  materials  is  restricted  to  specific  conditions  and  testing  windows  that 
are  monitored  by  NCSU-TOPS,  as  a  condition  of  this  contract  with  the  NCDPI.  All  test  materials  are  to  be 
disseminated  and  collected  timely  to  ensure  that  security  of  the  materials  is  maintained.  The  tests  that  are  eligible 
under  this  system  are  all  operational  end-of-course  tests  (as  appropriate). 

3.4  Administrative  Structure 

The  day-to-day  operations  of  the  North  Carolina  Testing  Program  are  managed  and  directed  by  the  Division  of 
Accountability  Services’  Test  Development  Section  and  Testing  Policy  and  Operations  Section.  The  Chief 
Consultant  for  Financial  Services  Purchasing  Section  and  other  personnel  in  the  NCDPI  will  oversee  the 
contractual  processes.  The  State  Board  of  Education  and  the  Office  of  Purchasing  and  Contracts,  Department  of 
Administration,  is  responsible  for  approving  contracts  and  contract  amendments.  The  Director  of  Accountability 
Services  will  be  the  liaison  between  NCSU-TOPS  and  the  financial  management  agencies  of  the  NCDPI.  NCDPI 
project  oversight  as  the  Project  Manager  for  the  NCSU-TOPS  contract  shall  be  the  responsibility  of  the  Chief 
Consultant  for  the  Test  Development  Section  who,  with  the  Director  of  Accountability  Services,  shall  work 
collaboratively  with  the  project  manager  at  NCSU-TOPS  to  ensure  that  the  terms  and  conditions  and  other 
requirements  of  this  contract  are  fully  implemented.  The  Chief  Consultant  for  the  Test  Development  Section  shall 
work  with  the  other  section  chiefs/  lead  consultants  and  the  Director  of  Accountability  Services  and  other  areas  of 
NCDPI  to  ensure  that  the  requirements  of  this  contract  are  fully  implemented. 

3.5  Enrollment 

Currently  there  are  1 15  public  school  systems  (Local  Education  Agencies-LEAs,  a  projected  151  public  charter 
schools,  approximately  7  special  public  schools,  approximately  50  participating  private  schools,  and  1  federal 
school  in  North  Carolina.  The  total  anticipated  2014-15  and  2015-16  public,  non-public,  and  federal  enrollments  in 
the  grades  (grades  3-12)  and  subjects  tested  as  part  of  the  North  Carolina  Testing  Program  is  approximately  1.6 
million,  with  many  students  at  certain  grades  taking  more  than  one  test. 
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3.6  Testing  Calendar 


North  Carolina  has  a  number  of  schools  operating  on  non-traditional  calendars,  specifically  a  4  x  4,  9-week,  or 
semester  block  schedule.  On  a  non-traditional  calendar,  students  complete  high  school  courses  in  one  semester 
rather  than  on  a  yearlong  basis.  This  has  necessitated  multiple  testing  dates  for  the  various  assessments  and 
requires  an  ever  increasing  effort  on  the  part  of  NCSU-TOPS  to  provide  appropriate,  quality  assurance 
management,  and  leadership  in  providing  the  support,  management,  and  security  which  will  ensure  successful 
implementation  of  the  revised  testing  and  accountability  programs.  A  copy  of  the  chart  depicting  the  proposed 
North  Carolina  Testing  Program  for  the  2014-15  and  2015-16  school  years  is  located  on  the  NCDPI  website  at 
www.ncpublicschools.org. 

3.7  Maintaining  Test  Security 

North  Carolina  tests  and  many  of  the  supporting  test  documents  are  secure  test  materials.  They  may  not  be  copied, 
reproduced,  or  discussed  except  with  personnel  that  have  a  legitimate  need  pursuant  to  carrying  out  the  terms  of 
this  contract.  Access  to  the  test  materials  shall  be  limited  to  personnel  with  a  legitimate  need.  Personnel  involved 
in  the  production,  assembly,  storage,  shipping  and  analyses  of  tests  and  test  data  may  not  share  the  content  of  the 
test  materials  or  the  data  except  as  directed  by  the  staff  of  the  NCDPI  Division  of  Accountability  Services. 
Personnel  involved  in  working  with  secure  test  materials  under  the  tenns  of  this  contract  may  not  share  documents 
with  LEAs,  schools,  or  other  public  or  private  entities  including  other  governmental  agencies  or  the  media  except 
as  directed  by  the  NCDPI  project  manager  of  this  contract.  All  public  requests  for  information,  including  the 
media  and  any  individual  other  than  district  test  coordinators,  must  be  directed  to  the  Chief  Consultant  for  the  Test 
Development  Section  Chief  or  the  Director  of  Accountability  Services.  NCSU-TOPS,  nor  any  of  its  employees, 
shall  not  respond  to  any  requests  for  information  or  make  any  comments  on  the  testing  and  accountability  program. 

Each  year  the  NCSU-TOPS  contract  project  manager  shall  develop  a  Test  Security  Agreement  that  is  approved  by 
the  Chief  Consultant  for  the  Test  Development  Section  Chief.  The  Test  Security  Agreement  must  (1)  state  that  the 
person  has  read  the  Test  Security  Agreement  and  the  North  Carolina  Testing  Code  of  Ethics  and  that  the  person 
understands  its  content  as  well  as  the  applicable  sanctions  in  the  event  a  violation  of  test  security  occurs.  The  Test 
Security  Agreement  must  be  read  and  signed  by  all  employees  (permanent  and  temporary).  The  contract  project 
manager  (NCSU-TOPS)  must  also  provide  all  employees  (permanent  and  temporary)  with  a  copy  of  the  North 
Carolina  Testing  Code  of  Ethics,  which  is  referenced  in  the  Test  Security  Agreement.  All  employees  (permanent 
and  temporary)  should  retain  a  copy  of  the  signed  test  security  agreement  and  the  North  Carolina  Testing  Code  of 
Ethics.  Each  new  employee  (full  time  or  temporary)  hired  to  work  on  the  project  shall  be  required  to  read  and  sign 
the  disclosure  form  at  the  time  of  employment  after  English  language  arts  the  North  Carolina  Testing  Code  of 
Ethics.  The  signed  Test  Security  Agreements  are  to  be  kept  on  file  by  the  NCSU-TOPS  project  manager  for 
reference  and  access,  if  needed,  by  the  NCDPI. 

The  facilities  that  are  used  to  assemble,  produce,  develop,  store,  analyze  and  ship  North  Carolina  tests  shall  have 
limited  access  for  individuals  not  involved  in  the  testing  process.  All  visitors  must  enter  through  the  front  main 
entrance  and  be  required  to  sign  in.  Every  effort  shall  be  made  to  ensure  that  visitors  are  escorted  during  their  time 
in  the  facility  and  are  not  able  to  acquire  access  to  secure  test  materials  or  data  files  at  any  stage  of  production  and 
delivery.  Visitors  will  be  required  to  wear  a  badge  which  clearly  identifies  them  as  visitors.  The  public  waiting 
area  shall  be  clearly  defined  with  signs  that  advise  that  visitors  are  not  allowed  beyond  identified  points.  Physical 
barriers,  to  the  extent  possible  and  feasible,  are  encouraged  to  further  define  the  visitors’  waiting  areas.  The  use  of 
the  facility  for  public  meetings  such  as  test  reviews  or  advisory  committee  meetings  shall  be  discouraged.  At  no 
time,  will  any  person  be  admitted  through  an  alternate  entrance,  such  as  the  warehouse  doors,  or  be  permitted 
access  to  secure  area  without  the  presence  of  NCDPI  staff  or  NCSU-TOPS  staff.  The  warehouse  bay  area  shall  be 
secured  at  all  times. 
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NCSU-TOPS  shall  account  for  all  test  materials  received  at  all  times.  Personnel  with  access  to  secure  North 
Carolina  Test  Materials  shall  not  use  their  access  to  those  materials  for  personal  gain.  NCSU-TOPS  shall  report  all 
incidents  of  test  security  violations  to  the  Chief  Consultant  for  the  Test  Development  Section  or  the  Director  of 
Accountability  Services. 

3.8  Electronic  Formatting  of  Tests  and  Test  Materials 

All  tests  and  test  items  (including  archived  tests  and  items  and  test  historical  data)  generated  for  the  North  Carolina 
Testing  Program  shall  be  stored,  managed,  and  maintained  electronically  using  a  database  system  that  will  facilitate 
being  imported  into  other  electronic  media,  as  required,  without  additional  individual  user  processing  or 
formatting.  Documents  including  tests  and  test  items,  to  the  extent  possible,  shall  be  created  using  a  system  that 
allows  for  management  with  database  and  web-based  applications  without  additional  formatting  or  processing. 
Tests  along  with  all  supporting  graphics  shall  be  stored  as  intact  electronic  files  so  that  intact  tests  can  be  retrieved, 
if  required.  NCSU-TOPS  shall  make  every  effort  to  minimize  the  amount  of  paper  used  to  support  the  statewide 
testing  program  by  continuously  exploring  ways  to  make  the  testing  operation  operate  more  effectively  and 
efficiently. 

3.9  Archiving  of  Documents  Noted  as  Public  Records 

Data  files,  documents,  reports,  tests,  and  other  items  generated  pursuant  to  this  contract  are  to  be  archived  and 
maintained  for  the  purpose  of  public  record  as  set  forth  by  the  Records  and  Retention  and  Disposition  Schedule  and 
Retention  Periods  governing  the  records  series  listed  in  Chapters  121  and  132  of  the  General  Statutes  of  North 
Carolina  and  the  General  Schedule  for  State  Agency  Records  issued  by  the  North  Carolina  Department  of  Cultural 
Resources  Office  of  Archives  and  History,  Division  of  Historical  Resources,  Archives  and  Records  Section, 
Government  Records  Branch  dated  January  10,  2004,  the  procedures  set  forth  under  N.  C.  Administrative  Code, 
Title  7,  Chapter  4,  Subchapter  M,  Section  500,  and  the  schedules  set  forth  by  the  NCDPI  Division  of 
Accountability  Services  acting  through  their  Records  Officer(s).  It  shall  be  the  responsibility  of  the  NCDPI 
Project  Manager  to  provide  guidance  to  the  project  manager  of  the  NCSU-TOPS  in  carrying  out  the  responsibilities 
of  records  retention  and  disposition  schedules  set  forth  by  North  Carolina  State  Government  as  they  are  applicable 
to  the  NCDPI  Division  of  Accountability  Services.  NCSU-TOPS  shall  be  responsible  for  retaining  a  minimum  of 
5  copies  of  any  and  all  documents  generated.  At  the  conclusion  of  the  contractual  relationship  between  the  NCSU- 
TOPS  and  NCDPI,  all  archived  data  files,  documents;  reports,  tests,  and  other  items  generated  pursuant  to  this 
contact  are  to  be  provided  to  the  NCDPI.  Conclusion  of  the  contractual  relationship  is  when  there  is  no  contract  in 
effect  between  NCSU-TOPS  and  the  NCDPI. 

3.10  Confidentiality  of  Student  Records 

Student  educational  records  are  subject  to  20  U.S.C.  1232g.  Family  Education  Rights  and  Privacy  Act  (FERPA) 
are  not  disclosable  except  in  very  limited  circumstances.  NCSU-TOPS  will  ensure  that  every  employee 
responsible  for  carrying  out  the  terms  of  this  contract  is  aware  of  the  confidentiality  requirements  of  federal  law.  In 
addition,  each  employee  must  sign  a  confidentiality  acknowledgement  that  indicates  that  he  or  she  understands  the 
legal  requirements  for  confidentiality.  NCSU-TOPS  is  responsible  for  the  actions  of  its  employees  and  must  take 
all  precautions  necessary  to  ensure  that  no  violations  occur.  Finally,  access  to  personally  identifiable  student 
education  information  will  be  limited  to  those  employees  who  are  required  to  have  access  in  order  to  perform  their 
responsibilities  pursuant  to  this  contract. 

3.11  Copyrighting  Test  Materials 
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The  staff  at  NCSU-TOPS  shall  be  responsible  for  assisting  the  NCDPI  staff  in  ensuring  that  all  created  tests  and 
supporting  materials  are  copyrighted  timely  and  hilly,  including  acquiring  trademarks  (if  needed)  for  the  actual 
names  of  the  tests  such  as  North  Carolina  End-of-Grade  Tests,  North  Carolina  End-of-Course  Tests, 
NCEXTEND1,  Grade  3  Reading  Portfolio,  NC  Final  Exams,  and  the  College  and  Career  Readiness  Alternates  for 
Grades  1 0  and  1 1 .  The  state  secure  tests  and  field  test  forms  and  other  secure  documents  are  to  be  copyrighted  in  a 
timely  manner  following  established  guidelines  of  the  Library  of  Congress. 

3.12  Use  of  Copyrighted  Materials 

The  test  development  staff  and  the  copyright  editor  at  NCSU-TOPS  shall  ensure  the  use  of  copyrighted  materials  is 
documented  and  that  the  appropriate  copyright  holder  is  identified  and  permission  to  use  is  sought  before  the 
materials  are  used.  NCSU-TOPS  staff  must  ensure  that  the  appropriate  documentation  is  filed  with  the  NCDPI 
noting  the  NCDPI  is  the  user  of  the  copyrighted  materials  and  that  any  fees  charged  are  incurred  by  the  NCDPI.  In 
addition,  NCSU-TOPS  shall  ensure  that  the  use  of  copyrighted  materials  follows  established  guidelines,  relative 
rates,  and  percentages  used  as  opposed  to  public  domain  material  and  commissioned  work.  Content  staff  must  be 
made  fully  aware  of  guidelines  for  use  of  copyrighted  materials  including  the  right  to  deny  use  if  rates  exceed  an 
established  reasonable  amount. 

3.13  Delivering  Selected  Tests  Online 

It  shall  be  the  expectation  that  tests  designated  by  the  department  will  be  delivered  online  either  as  the  sole 
mechanism  for  test  delivery  or  as  an  option.  NCSU-TOPS  staff  shall  have  the  responsibility  for  providing  the 
necessary  programming  and  hosting  services  as  well  as  technical  support  services  to  deliver  online  assessments. 

NCSU-TOPS  must  use  LEA/school  supplied  student  registrations  and  sessions  scheduled  data  to  create 
participation  forecasts  to  project  online  testing  volume  on  any  given  day  during  the  testing  window.  NCDPI  must 
be  able  to  view  reports  of  the  projected  online  testing  volume  at  any  time  starting  45  days  prior  to  and  throughout 
the  test  administration  window.  All  online  delivery  failures  must  be  documented  and  provided  to  NCDPI  daily 
during  the  testing  window,  and  a  summary  report  must  be  provided  within  45  days  of  the  last  day  of  the  testing 
window. 
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Section  IV.  Scope-of-Work 


4.1  Test  Development  Services  and  Psychometric  Technical  Assistance  and  Support  (60%): 

4.1.1  Scope  of  Work 

The  function  and  tasks  required  of  this  work  group  are  to  engage  in  the  activities  and  provide  psychometric, 
statistical,  and  technical  support  necessary  to  develop  the  assessment  instruments,  both  paper/pencil  and  online  and 
general  and  alternative  assessments,  required  to  successfully  deliver  the  North  Carolina  Testing  Program.  This 
work  group  is  also  responsible  for  the  design  and  development  of  the  NC  Final  Exams,  assessments  for  the  non- 
tested  subjects  used  for  teacher  effectiveness  measures.  In  addition,  NCSU-TOPS  is  to  assist  in  the  design  and 
implementation  of  a  comprehensive  assessment  system  to  meet  the  needs  of  students  including  technical  assistance 
and  support  for  an  on-going  classroom  based  online  assessment  tool  and  test  item  bank  (at  both  school/LEA  level). 
This  work  unit  must  possess  the  professional  and  technical  skills  essential  to  effectively  and  efficiently 
oversee/manage  and  provide  quality  control  measures  that  assure  efficiency  in  the  implementation  of  the  test 
development  operation  performed  by  the  measurement  and  scoring  specialists  to  produce  assessment  instruments. 
This  function  is  a  major  component  of  the  contract  scope-of-work  and  represents  approximately  60  percent  of  the 
resources  (personnel  and  budget)  required  to  implement  the  work  described  in  this  project  and  necessary  for  the 
successful  implementation  of  North  Carolina’s  testing  and  accountability  programs.  A  major  responsibility  of  this 
group  is  to  provide  support  in  producing  technically-sound,  valid,  and  reliable  tests  that  are  grammatically  correct 
and  contain  the  correct  answer  keys  that  utilize  current  technology  systems,  Principles  of  Universal  Design,  and 
general  assessment  standards  required  by  tire  USED  Peer  Review  Process  and  Joint  Committee  Standards  (APA 
and  AERA). 

The  major  tasks  and  activities  currently  identified  for  this  component  of  the  contract  are  listed  and  are  described  in 
detail  below: 

1)  Assist  NCDPI  staff  in  coordinating  the  activities  required  to  develop  assessments  by  (1)  developing  and 
maintaining  an  online  training  system  for  item  writing  and  reviewing,  (2)  recruiting  item  writers 
(teachers),  developing  item  specifications,  (3)  writing  and  developing  assessment  items,  (4)  editing  and 
revising  existing  tests,  and  test  items,  (5)  coordinating  advisory  reviews,  (6)  inputting  test  items  into 
testing  program  databases,  (7)  creating  test  items  artwork  as  needed,  and  (8)  developing  and 
maintaining  answer  keys  required  for  psychometric  analyses.  In  addition  to  partnering  with  the 
Division  of  Accountability  Services,  the  fulfillment  of  this  work  requires  NCSU-TOPS  to  collaborate 
with  and  support  the  work  of  the  NCDPI  Curriculum  and  Instruction  staff,  the  NCDPI  Exceptional 
Children’s  staff,  and  the  NCPI  English  as  a  Second  Language  staff. 

2)  Assist  NCDPI  staff  with  coordinating  the  activities  required  to  conduct  item  tryouts  and  field  tests  as 
needed  to  maintain  the  appropriate  number  of  assessment  instruments  (this  may  be  an  ongoing  process 
as  field  test  items  are  embedded  in  the  operational  tests  each  year).  These  activities  include,  but  are  not 
limited  to  (1)  securing  LEA  participation,  (2)  assisting  in  the  development  of  alternative  field  test 
models,  (3)  developing  and  producing  field  tests,  (4)  developing  and  producing  answer  sheets,  (4) 
producing  test  administrator’s  manuals,  (5)  assisting  with  the  production  of  LEA  field  test  training 
materials,  (6)  coordinating  the  packaging,  shipping,  and  return  of  field  test  materials,  (7)  receiving  and 
checking  completed  field  test  answer  documents,  (9)  and,  scanning/scoring  field  test  documents. 

3)  Assist  NCDPI  staff  with  analysis  of  field  test  data  by  generating/reviewing  item  statistics  and 
developing  reporting  statistics. 
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4)  Assist  NCDPI  staff  with  generating,  modifying,  verifying,  and  checking  the  NCDPI  specific  computer 
software  code  required  to  maintain  the  scanning/scoring/report  producing  program  and  other  computer 
programs  specific  to  the  development  component  of  the  North  Carolina  Testing  Program. 

5)  Analyze  item  data  and  obtain  answer  key  verifications  for  all  tests  included  in  the  North  Carolina 
Testing  Program  to  facilitate  decisions  regarding  production  of  test  forms. 

6)  Provide  technical,  professional,  and  clerical  assistance  to  maintain  test  development  item  archives,  item 
and  test  records,  statistical  references,  and  field  test  data.  Assist  with  item  production,  which  includes, 
but  is  not  limited  to  inputting  data  from  curriculum  surveys  into  appropriate  databases,  preparing  item 
records,  editing  and  proof  English  language  arts  items  and  tests. 

7)  Provide  professional,  technical,  and  curricular  consultation  services  in  test  development,  scoring,  and 
reporting  of  assessments. 

8)  Assist  with  providing  technical  and  administrative  support  for  performance  assessment  requests  for 
proposals  (RFPs)  required  to  implement  the  performance  testing  component  of  the  NC  Testing 
Program. 

9)  Assist  NCDPI  staff  by  providing  an  electronic  database  for  the  item  pool  that  is  used  to  assemble  North 
Carolina  tests  ensuring  that  the  test  item  pool  is  compatible  with  online  testing  project  requirements. 

1 0)  Assist  NCDPI  staff  with  actual  test  assembly  of  all  tests  included  in  the  NC  testing  program  including 
traditional,  non-traditional,  general,  alternative,  and  alternate  assessments. 

1 1)  Assist  NCDPI  staff  in  identifying  and  subcontracting  with  prospective  item  writing  and  item 
development  vendors  which  shall  to  a  great  degree  include  North  Carolina  teachers. 

12)  Assist  NCDPI  in  conducting  the  appropriate  validation  studies  including  data  collection  and  analyses 
required  to  ensure  technical  soundness  of  North  Carolina  state  developed  tests. 

13)  Assist  NCDPI  in  the  annual  production  of  test  administration  manuals  for  each  component  of  the  North 
Carolina  Testing  Program  by  editing  the  directions  for  administering  the  tests  and  ensuring  their 
alignment  with  the  test  design  as  required. 

14)  Assist  with  the  development  of  the  design  and  creation  of  technical  reports  to  provide  technical 
information  pertinent  to  the  development  of  the  mathematics,  science,  English  language  arts,  and  social 
studies  tests. 

1 5)  Assist  with  a  design  of  tests  and  test  items  to  facilitate  continuation  of  moving  forward  with 
administering  designated  tests  online. 

1 6)  Support,  maintain,  and  enhance  the  electronic  system  for  generating,  managing,  storing,  and  archiving 
test  items. 

1 7)  Assist  NCDPI  staff  in  the  generation  of  test  specifications  and  blueprints  for  each  individual  test  to  be 
developed. 

1 8)  Create  and  manage  large  scale  online  distributed  scoring  system  for  scoring  constructed  response  items. 
This  system  includes  a  large  scale  imaging  and  data  collection,  rescoring  for  reliability  analysis 
reporting,  as  well  as  post  processing  and  final  score  inclusion.  Conduct  scoring  via  this  system  for  ten 


12 


NCFE  courses.  Final  scores  delivered  to  NCDPI  within  extremely  short  time  frames  so  that  students  can 
receive  scores  at  the  end  of  the  instructional  period. 

4.1.2  Contractor  Proposes  Budget 

NCSU-TOPS  is  to  propose  an  operational  budget  for  the  2014-15  and  2015-16  fiscal  years  for  this  component  of 
the  contractual  agreement. 

4.2  Technical  Assistance  and  Support,  Programming  Services,  Logistical  Solutions,  Test  Production  and 
Layout,  Warehousing  and  Materials  Delivery,  Online  Testing  Delivery  and  Hosting,  and  Archiving 
of  Documents  (40%) 

4.2.1  Scope  of  Work 

The  function  and  tasks  required  of  this  work  group  shall  be  to  use  a  variety  of  sophisticated  data  collection  and 
statistical  analysis  tools  to  ensure  that  all  data  collected  and  reported  as  part  of  the  SBE’s  school-level 
accountability  and  testing  programs  are  accounted  for,  appropriate,  and  correct.  Extremely  important  decisions 
concerning  the  operation,  organization,  and  perfonnance  of  North  Carolina’s  approximately  2,500  public  schools 
will  be  based  on  these  data.  The  group  will  select  statistically  appropriate  student  and/or  school  samples  for  field 
tests  and  for  the  administration  of  sampled  tests.  The  staff  shall  also  institute  a  system  of  quality  control 
procedures  to  ensure  the  integrity  and  quality  of  the  data.  The  group  will  carry-out  data  collection  design,  data 
audits,  data  checks,  and  will  produce  the  State  Testing  Results  (The  Green  Book)  which  shall  include  results  from 
all  state  mandated  assessments  and  alternate  assessments. 

The  duties/work  will  include  designing,  defining,  establishing,  editing,  and  completing  the  data  collection 
procedures  essential  to  ensuring  that  test  and  related  data  shall  be  collected  in  an  appropriate  and  correct  mamier 
with  100  percent  accuracy.  The  staff  shall  work  with  the  NCDPI  staff  in  all  of  these  endeavors  as  directed.  The 
remaining  time  should  be  devoted  to  conducting  data  analysis  and  creating  electronic  tools  for  data  analysis  that 
result  in  statistically  and  psychometrically  accurate  and  timely  reports  both  in  paper  formats  and  as  web-based 
tools.  In  summary,  the  purpose  of  this  professional  work  group  is  to  provide  the  work/services  (e.g.,  quality 
control  and  data  analysis)  typically  performed  by  an  applied  research  group.  This  work  group  component  of  the 
contract  represents  approximately  8  percent  of  the  total  contract  and  must  be  staffed  with  experienced,  technically 
trained  statistical  analysts,  researchers,  data  analysts,  quality  control  managers  and  statisticians.  The  major  tasks 
and  activities  currently  identified  for  this  component  of  the  contract  are  listed  and  described  in  detail  below: 

4.2.2  Use  Statistical  Software  Packages  to  Conduct  Data  Analyses 

NCSU-TOPS  is  to  use  SAS,  and  other  commercially  available  sophisticated  statistical  software  packages  and 
computer  programs,  online  software  program  editors,  and  graphing  packages  to  prepare  special,  in-depth  analysis 
related  to  school-level  accountability  efforts  for  end-of-grade  (EOG),  end-of-course  (EOC),  alternate  assessments, 
field  tests,  special  studies,  and  other  test  data  per  the  direction  of  the  NCDPI  Accountability  Services/Test 
Development  staff.  NCSU-TOPS  must  analyze  testing  and  accountability  data  and/or  data  gathering  models  as 
necessary  to  ensure  that  appropriate  statistical  procedures  are  being  applied  to  the  generation,  collection,  and 
combining  of  the  testing  and  accountability  data.  As  part  of  the  analysis  process,  NCSU-TOPS  must  verify  data 
files  submitted  to  the  NCDPI  and  from  other  divisions  and  agencies  for  data  accuracy,  integrity,  and  correctness. 
NCSU-TOPS  is  expected  to  upgrade  the  statistical  software  as  required  to  ensure  efficiency  and  compatibility.  The 
NCSU-TOPS  project  manager  is  to  inform  the  NCDPI  project  manager  any  time  that  NCSU-TOPS  decides  to 
upgrade  the  statistical  software  packages  since  compatibility  between  the  two  organizations  is  critical  for  the 
efficiency  and  success  of  the  program  operation. 
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4.2.3  Assist  with  the  Processing  of  LEA  Data 

NCSU-TOPS  is  to  assist  as  is  appropriate  and  as  directed  with  the  processing  of  test  data  files  received  from  115 
LEAs  and/or  their  public  schools  and  public  charter  schools  by  serving  as  a  quality  control  check  to  ensure 
accuracy  of  the  data  from  the  paper  and  pencil  and  online  assessments.  NCSU-TOPS  is  to  consolidate  LEA  and/or 
school-level  data  files  using  specially  developed  NCDPI/NCSU-TOPS  programs  to  produce  NCDPI  state-level 
files  for  the  State's  accountability  programs.  To  verify  accuracy  of  aggregating  processes,  NCSU-TOPS  will  merge 
programs  by  instituting  appropriate  quality  control  procedures. 

NCSU-TOPS  will  produce  special  analysis  on  policy  issues  emerging  from  State  testing  programs  (e.g.,  end-of- 
grade,  end-of-course,  and  alternates),  State  accountability  programs,  and  other  SBE  requests  as  required. 

Specifically,  NCSU-TOPS  shall: 

1)  assist  by  supporting  tasks  and  activities,  as  required,  to  facilitate  the  design  and  redesign  of  tests  to  be 
administered  online, 

2)  ensure  data  collected  online  for  the  tests  undergo  a  quality  assurance  procedure  before  data  are  exported 
for  processing  and  analysis, 

3)  all  data  output  and  reports  generated  by  the  terms  of  this  contract  are  not  to  be  reported  or  disseminated 
until  approved  by  the  project  manager  from  NCDPI, 

4)  Facilitate  secure  data  collection  and  analysis  processing  by  providing  technical  support  services  to  staff 
and  to  NCDPI,  including  but  not  limited  to 

a.  VPN  connected  services; 

b.  SSH  hosting  capacity  used  to  transfer  LEA  data; 

c.  SAS  services  to  run  ad-hoc  and  pre-programmed  analysis  with  an  emphasis  on  tighter  business 
process  integration; 

d.  End-user  technical  support  through  help  desk  type  operations;  and, 

e.  Data  storage  and  database  services. 

4.2.4  Contractor  Proposes  Budget  for  Analysis  Group 

NCSU-TOPS  is  to  propose  an  operational  budget  for  the  2014-15  and  2015-16  fiscal  years  for  this  component  of 
the  contractual  agreement. 

4.3  Research  Design  and  Studies,  Data  Collection  and  Processing,  Statistical  Analyses,  Report 
Generation,  and  Evaluation  Support  (8%) 

4.3.1  Scope  of  Work 

This  work  group  shall  be  responsible  for  (1)  the  efficient  and  timely  layout,  mock-up  and  production  of  all  tests  and 
supporting  test  materials,  (2)  the  printing,  checking  and  storage  of  tests  and  test  materials,  and  (3)  the  receiving, 
handling  (pick/pack),  shipment  and  return  of  test  materials  to  and  from  the  school  systems.  In  addition  to  the 
regular  shipping  and  handling  of  the  State-required  tests,  this  unit  shall  design  and  institute  an  electronic  system  to 
take  orders  for  the  State  testing  program  and  support  a  similar  warehousing  operation  for  the  non-public  or  private 
schools  or  other  state  educational  institutions  such  as  the  Department  of  Health  and  Human  Services,  Department 
of  Juvenile  Justice  and  Delinquency  Prevention,  Federal  Schools,  and  the  Department  of  Community  Colleges 
interested  in  the  procurement  of  North  Carolina’s  tests.  This  work  unit  will  need  to  be  able  to  receive  and  process 
orders  for  the  various  test  materials  on  a  daily  basis  and  be  able  to  ship  the  requested  test  materials  in  a  timely 
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manner  to  the  appropriate  LEAs  and  schools  utilizing  warehousing  facilities  based  on  the  warehousing  needs  of  the 
North  Carolina  Testing  Program  as  identified  from  operational  data  for  the  2014-15  and  2015-16  school  years. 

The  personnel  employed  to  staff  the  technical,  maintenance,  support  and  programming  component  of  this  work 
group  must  possess  a  variety  of  highly  technical  computer  skills  and  computer  related  skills,  must  be  able  to  solve 
a  diverse  number  and  type  of  computer  hardware  and  software  problems,  some  of  which  are  typical  along  with 
problems  which  are  unique  to  this  testing  program  and  test  scoring  operation. 

1)  Assist  NCDPI  staff  with  the  development  and  production  of  final  versions  of  assessment  instruments. 
Such  activities  include,  but  are  not  limited  to,  developing  answer  sheet  as  approved  and  test  book 
layouts,  production  of  final  PDF  for  printing,  and  edit/quality-control  check  all  final  test  document 
production  activities  and  products,  including  web-based  assessment  data  collection  projects. 

2)  Assist  NCDPI  staff  with  overseeing  the  printing  and  production  of  testing  and  accountability  test 
materials.  Such  activities  include,  but  are  not  limited  to,  quality-control  checking  proofs  for  test  books, 
answer  sheets,  and  administrator’s  manuals,  and  overseeing/monitoring  the  printing  contractor’s 
printing  operations  and  receiving/inventorying  printed  test  materials. 

3)  Provide  printing  services  for  (1)  large-print  and  one-item  per  page  test  materials  for  EOGs,  EOCs, 
North  Carolina  Final  Exams,  Grade  3  Reading  Portfolio  and  NCEXTEND1  and  (2)  accommodations 
monitoring  forms  for  all  assessments. 

4)  Manage  and  operate  test  materials  receiving,  order  fulfillment,  and  delivery  warehousing  facility.  Such 
activities  include,  but  are  not  limited  to:  receiving/logging-in  test  material  shipments  from  the  printing 
contractor  and  related  vendors,  receiving/processing  material  orders,  operating  a  pick/pack  materials 
distribution  center,  packaging  materials  for  shipment,  maintaining  a  materials  inventory  system, 
managing  the  delivery  of  the  test  materials  to  the  appropriate  customers  (e.g.,  LEAs). 

5)  Improve  the  overall  cost-effectiveness  and  efficiency  of  the  tracking  procedures  for  test  materials 
handling/shipping  by  implementing  a  test  materials  receiving,  pick/pack,  and  distribution  operation 
located  at  a  central  North  Carolina  site  utilizing  the  most  current  technology. 

6)  Streamline  and  improve  test  materials  handling  procedures  including  receiving  and  inventorying  test 
books,  answer  documents,  and  related  testing  program  printed  materials/data  files.  Prepackage  and/or 
package  testing  program  and  test  utilization  materials  for  LEAs  and  special  schools  and  school  systems 
and  other  NCDPI  customers.  Prepare  packing  lists  and  related  material  shipping  documents  from 
information  provided  by  LEAs  through  the  online  ordering  system.  Package  and  ship  LEA  test 
materials  according  to  packing  lists  and  testing  program  specifications. 

7)  Design  and  maintain  an  online  ordering  system  for  LEAs  to  order  test  materials  for  each  testing  cycle 
(fall,  spring,  and  summer).  The  ordering  system  must  provide  LEAs  with  a  projected  membership  for 
each  grade  or  test  population  as  necessary  to  achieve  order  quantities.  Work  to  integrate  membership 
data  from  LEAs  into  the  order  system  with  the  goal  of  reducing  manually  entered  orders. 

8)  Assist  NCDPI  staff  in  defining  the  specifications  for  shipping  cartons,  pallet  layout  and  materials 
package  for  efficient  storage  and  shipping. 

9)  Assist  in  writing  Requests  for  Proposals  (RFPs)  as  required  and  evaluating  subsequent  proposals  when 
further  outsourcing  of  services  is  required. 
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1 0)  Assist  in  the  development  of  specifications,  acquisition,  and  maintenance  of  the  hardware  and  software 
necessary  for  the  support  of  the  intemal/extemal  systems  including  the  computer-based  systems 
required  for  the  implementation  of  the  North  Carolina  Testing  Program 

11)  Provide  technical  support  for  the  design  of  the  divisional  web  site(s)  and  to  provide  for  the  upload  of 
files  to  the  web  site  as  requested. 

12)  Develop,  maintain,  and  enhance  as  needed,  the  Testing  News  Network  (TNN)  to  provide  an  electronic 
medium  for  the  dissemination  of  secure  testing  infonnation,  test  files,  and  other  secure  documents  and 
applications  to  LEA  test  coordinators  and  their  superintendents. 

13)  Maintain  and  continue  to  explore  additional  ways  to  improve  the  tracking  system  for  managing  the 
warehouse  inventory  of  test  materials. 

14)  Create  and  oversee  Beta  testing  and  quality  control  processes  as  required  for  any  technology  based 
assessments. 

15)  Assist  with  the  programming  needs  of  the  statewide  testing  and  accountability  programs. 

16)  Assist  the  NCDPI  with  the  online  test  design,  item  formatting,  and  programming. 

1 7)  Assist  with  the  online  delivery  of  the  alternate  assessments  and  prediction  of  number  of  participants. 

18)  Host  the  technological  delivery  of  online  test  administrations  and  data  collection  projects  such  as  EOC 
online,  NCEXTEND1,  NC  Final  Exams  scoring  and  other  projects  as  needed. 

1 9)  Maintain  a  system  for  electronic  archiving  of  all  documents  designated  as  public  record  for  purpose  of 
retention  and  access. 

20)  Continue  to  explore  ways  to  reduce  the  footprints  and  bandwidth  needs  of  the  design  of  each  of  the 
online  testing  projects. 

21)  Provide  the  NCDPI  and  the  public  a  high  level  of  assurance  that  test  security  is  maintained  continuously 
for  all  testing  projects  and  in  particular  for  web-based  delivered  testing  projects  ensuring  that  the 
security  and  confidentiality  of  web  delivered  test  items  and  subsequent  student  records  cannot  be  easily 
compromised. 

22)  Assist  the  NCDPI  accountability  with  application  reengineering,  development,  testing,  hosting,  and 
deployment  for  the  redesign  of  the  Accountability  application  suite  including  ABCTools,  Winscan, 
Individual  Report,  Accountability  Database,  etc. 

4.3.2  Contractor  Proposes  a  Budget 

NCSU-TOPS  is  to  propose  an  operational  budget  for  the  2014-15  and  2015-16  fiscal  years  for  this  component  of 
the  contractual  agreement.  4.3  Research  design  and  studies,  Data  Collection  and  Processing,  Statistical  Analyses, 
Report  Generation,  and  Evaluation  Support  (8%); 
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4.4.1  Scope  of  Work  Summary 


The  NCDPI  has  made  every  attempt  to  describe  in  detail  the  total  scope-of-work  required  to  be  performed  under 
this  contract;  however,  the  very  nature  of  defining  and  implementing  a  statewide  testing  and  accountability 
model/program  requires  adjustments  to  the  scope-of-work  from  time  to  time.  Therefore,  the  NCDPI  reserves  the 
right  to  redefine,  to  assume  the  responsibility  of  the  tasks,  and/or  to  add  to  the  tasks  which  must  be  performed  to 
successfully  support  the  implementation  of  the  North  Carolina  Testing  Program  as  per  contract  negotiations 
specifications  contained  in  the  “Terms  and  Conditions”  of  the  Contract  Agreement  permit.  In  addition,  the  NCDPI 
project  coordinator  will  work  with  the  NCSU-TOPS  contract  manager  to  produce  budget  tracking  and  monitoring 
methodology  that  will  yield  per  test  costs  to  be  consistent  with  the  practices  mandated  at  the  NCDPI  in  order  to 
project  budgetary  expenditures  and  requirements  by  test. 

5. 0  Timeline 

All  activities  shall  be  completed  according  to  the  timelines  established  by  the  NCDPI  Division  of  Accountability 
Services  staff  and  conveyed  to  the  contract  project  coordinator  named  in  NCSU-TOPS ’s  proposal  prior  to  the 
execution  of  the  contract  Agreement.  The  NCDPI  reserves  the  right  to  modify  the  products  and  services  delivery 
timelines  as  needed  to  carry  out  the  directives  of  the  SBE  and  the  General  Assembly.  Such  revisions  in  project 
schedules  shall  be  mutually  agreed  to  by  and  between  the  Division  of  Accountability  Services  staff  and  NCSU- 
TOPS’s  project  coordinator  prior  to  inclusion  in  the  project’s  timeline  specifications  and  such  modifications  shall 
be  subject  to  the  provisions  contained  in  the  “Terms  and  Conditions”  section  of  the  Contract  Agreement. 

All  activities  to  be  completed  under  this  Contract  Agreement  shall  be  completed  on  or  before  June  30,  2016. 

6.0  Program  Monitoring  and  Progress  Report  Meetings 
6.1  Assignment  of  NCSU-TOPS  Project  Manager 

NCSU-TOPS  will  designate  one  staff  person  to  work  closely  with  the  NCDPI  coordinator  for  this  project.  All 
questions  and  correspondence  concerning  the  project  should  be  between  these  respective  representatives  in  order  to 
ensure  effective  coordination  and  communication  between  NCSU-TOPS  and  the  NCDPI.  Also,  because  of  the 
differences  in  tasks,  NCSU-TOPS  should  designate  Project  Directors  for  each  of  the  major  project  components: 
Editing,  Production,  Test  Development,  Statistical  Analysis/Reporting,  and  Warehouse.  The  NCDPI  project 
coordinator  is  also  accountable  to  other  divisions  of  the  NCDPI  (Administrative  Services,  Contractual  Services, 
etc.)  for  this  project  and  will  have  full  sign-off  authority.  The  NCDPI  requests  that  their  coordinator  be  authorized 
to  work  directly  with  NCSU-TOPS's  department  directors  and  managers  in  order  to  expedite  the  sign-off  of  all 
project  work. 

Because  of  the  dynamic  and  evolving  nature  of  the  testing  and  accountability  programs,  the  coordinators  must  be 
authorized  to  take  advantage  of  unforeseen  opportunities  to  provide  the  best  possible  product  and  services  for 
North  Carolina’s  public  schools.  Such  revisions  will  be  incorporated  into  the  project  if  both  coordinators  agree  in 
writing  to  the  changes  and  if  such  modifications  can  be  accomplished  with  available  resources.  A  final  accounting 
of  all  program  modifications  will  be  prepared  before  and  presented  at  the  final  progress  report  meeting. 

NCSU-TOPS  should  anticipate  at  least  quarterly  progress  meetings.  For  planning  purposes,  the  initial  meeting  will 
occur  immediately  after  the  contract  is  awarded.  The  parties  will  meet  quarterly  or  as  frequently  as  is  necessary  for 
progress  reporting  and  resource  allocation  monitoring.  At  the  final  session  the  entire  project  will  be  critiqued  and 
suggestions  for  improvement  made. 
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7.0  Payment  Schedule 


NCSU-TOPS  shall  invoice  the  NCDPI  monthly,  according  to  the  provisions  set  forth  in  the  “Terms  and 
Conditions”  section  of  the  Contract  Agreement.  Any  unused  funds  will  revert  to  the  NCDPI. 

8.0  Proposal  Evaluation  Criteria 

The  following  five  evaluation  criteria  will  be  used  to  determine  the  responsiveness  of  NCSU-TOPS’s  proposal  to 
the  RFP  and  are  arranged  in  descending  order  of  importance  to  the  NCDPI. 

8.1  Quality  Control  Procedures 

Due  to  the  detailed  nature  of  the  RFP,  the  NCDPI  will  focus  on  contractor  detail  on  those  procedures  that  ensure 
the  products  and  services  described  are  delivered  without  error  or  need  for  NCDPI  staff  time  to  resolve  problems. 
NCSU-TOPS  should  emphasize  those  measures  that  are  beyond  normal  industry  standards.  Such  measures  should 
include,  but  are  not  limited  to,  the  following:  (1)  key  checks  of  all  camera-ready  copies  of  tests,  (2  verification  of 
all  items  on  the  camera-ready  copy  with  the  item  record,  (3)  verification  of  the  camera-ready  copy  with  the  digital 
proof  from  the  printer,  (4)  check-in  process  of  20%  of  the  shipments  from  the  printer  to  verify  the  product  has 
been  printed  with  the  correct  number  of  pages  and  with  pages  collated  in  the  correct  order,  and  (5)  verification  of 
the  accuracy  of  shipments  to  LEAs  by  a  person  other  than  the  person  who  packed  the  order. 

NCSU-TOPS  must  ensure  day-to-day  security  to  prohibit  unauthorized  personnel  access  to  North  Carolina  Testing 
Program  test  materials  through  deliberate  or  unintentional  actions.  Security  also  refers  to  and  includes  oral 
communication  by  staff  of  the  particular  day-to-day  operations  and  content  of  test  materials  to  the  general  public. 
NCSU-TOPS  is  held  responsible  for  the  actions  of  the  staff  and  must  guard  against  a  breach  of  security.  Staff  must 
be  carefully  screened  and  made  to  understand  the  importance  of  security,  hi  addition,  each  staff  must  sign  a  non¬ 
disclosure  document  attesting  to  their  understanding  of  the  expected  behavior  in  not  discussing  or  revealing  the 
contents  of  the  tests  and  the  results  from  the  tests.  NCSU-TOPS  is  responsible  for  creating  the  non-disclosure 
document  but  must  ensure  that  it  meets  the  NCDPI  standards  and  specifications.  The  facility  should  be  structured 
in  a  manner  that  prohibits  the  general  public  access  to  secure  test  materials. 

8.2  Contractor's  Competence 

The  NCDPI  will  thoroughly  review:  (a)  the  qualifications  of  the  prospective  contractor's  personnel  assigned  to  this 
project,  (b)  experience  with  the  tasks  for  which  each  person  will  be  responsible,  and  (c)  the  estimated  time  each 
will  be  involved  in  the  project. 

8.3  Feasibility  and  Cost  Effectiveness  of  Contractor's  Plan 

The  methods  proposed  to  accomplish  each  task  will  be  reviewed  for  specificity,  feasibility,  schedule,  plans  for 
scoring,  and  plans  for  data  entry,  cost/benefit,  and  consistency  with  NCDPI  defined  scope-of-work  and  program 
objectives. 

8.4  Overall  Cost 

The  cost  associated  with  carrying  out  the  terms  of  this  contract  shall  be  categorized  according  to  the  three 
components  of  the  contract.  Cost  related  to  personnel  shall  be  specified.  The  overhead  costs  related  to  the 
execution  of  this  contract  shall  not  exceed  1 0  percent  of  the  total  contract  costs. 
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8.5  Fulfillment  of  RFP  Requirements 


NCSU-TOPS  is  expected  to  comply  with  all  requirements  of  the  RFP  within  the  given  time  frame.  Previous 
experience  with  similar  tasks  will  be  considered. 

8.6  Delivery  of  Service  on  Time 

Proposals  should  include  a  description  of  any  previous  similar  work  perfonned  by  the  prospective  contractor.  The 
NCDPI  may  contact  agencies  that  have  worked  or  are  working  with  the  prospective  contractor. 

9.0  Contract  Information 

Three  copies  of  NCSU-TOPS ’s  response  to  this  RFP  must  delivered  on  or  before  2:00  p.m.  EST,  Wednesday  May 
23, 2014,  to  Dr.  Tammy  Howard,  Director  of  Accountability  Services,  North  Carolina  Department  of  Public 
Instruction,  NC  Education  Building,  301  N.  Wilmington  Street,  Raleigh,  North  Carolina  27601-2825. 

NCSU-TOPS  will  need  to  include  in  its  proposal  a  description  of  the  proposed  organization/personnel  proposed  to 
accomplish  the  project  tasks.  Pages  of  the  proposal  must  be  numbered  and  the  document  clearly  labeled  on  the! 
outside,  “Proposal:  Technical  and  Coordination  Support  for  the  North  Carolina  Testing  and  Accountability 
Programs.”  Questions  concerning  this  RFP  should  be  addressed  to  Tammy  Howard,  Division  of  Accountabilip 
Seryiees^  MCDPI,  6314  Mail  Service  Center,  Raleigh,  North  Carolina  27699-  6314,  and  must  be  received  by  1  :QQ 
p.m.  Friday,  May  10,  2014,  so  that  the  potential  contractor  can  receive  a  written  copy  of  questions  and  answers  td 
questions  prior  to  the  deadline  for  proposals.  Because  this  is  a  contract  between  a  state  agency  and  the  State 
University  system,  a  task  order  agreement  must  be  processed  in  order  to  fully  implement  this  contract, 

The  State  of  North  Carolina  requires  that  any  contract  between  state  agencies  meet  certain  general  terms  and 
conditions.  The  terms  and  conditions  as  they  relate  to  this  project  between  the  North  Carolina  Department  of 
Public  Instruction  (NCDPI)  and  NCSU-TOPS  are  listed  below: 

9.1  Contract  Terms  and  Conditions 

1)  Contract  Residency.  It  is  agreed  between  the  parties  hereto  that  the  place  of  this  contract,  its  situs  and  forum, 
will  be  Wake  County,  North  Carolina,  and  in  said  county  and  state  will  all  matters,  whether  sounding  in 
contract  or  tort  relating  to  the  validity,  construction,  interpretation  and  enforcement  of  this  contract,  be 
determined.  North  Carolina  law  will  govern  the  interpretation  and  construction  of  this  contract. 

2)  Project  Coordinator.  The  Project  Coordinator  shall  be  responsible  for  ensuring  Contractor  conformance  with 
the  terms,  conditions,  performance  specifications  as  set  forth  in  this  contract,  and  an  evaluation  of  NCSU- 
TOPS’s  performance.  NCSBE  has  complete  discretion  in  replacing  the  Project  Coordinator  with  another 
person  of  its  own  choosing. 

3)  Right  of  Termination.  NCSBE  may  terminate  this  agreement  at  any  time  at  its  complete  discretion  by  thirty 
days  written  notice  from  NCSBE  to  NCSU-TOPS.  In  that  event,  all  finished  or  unfinished  documents  and 
other  materials  shall,  at  the  option  of  NCSBE,  become  its  property.  If  the  contract  is  terminated  by  NCSBE, 
as  provided  herein,  NCSU-TOPS  will  be  paid  in  an  amount  which  bears  the  same  ratio  to  the  total 
compensation  as  does  the  service  actually  performed  to  the  total  service  set  forth  in  this  agreement. 

4)  Method  of  Payment.  NCSU-TOPS  will  submit  an  invoice  or  invoices  according  to  the  payment  schedule  set 
forth  in  this  agreement.  Invoices  should  be  billed  to  the  Payables  Section,  North  Carolina  Department  of 
Public  Instruction,  but  directed  to  the  Project  Coordinator,  as  identified  herein,  for  approval  of  payment. 
Payment  will  be  made  within  fifteen  days  upon  receipt  by  the  Payables  Section  of  a  requisition  for  payment 
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from  the  Project  Coordinator  confirming  that  NCSU-TOPS  has  satisfactorily  completed  the  work  required 
under  this  contract  and  evidenced  by  said  invoices. 

5)  Contract  Funding.  It  is  understood  and  agreed  between  NCSU-TOPS  and  NCSBE  that  the  payment  of 
compensation  specified  in  this  agreement,  its  continuation  or  any  renewal  or  extension  thereof,  is  dependent 
upon  and  subject  to  the  appropriation  of  funds  to,  and  allocation  by,  NCSBE  for  the  purpose  set  forth  in  this 
contract.  Should  such  funds  not  be  appropriated  or  allocated,  this  contract  shall  immediately  be  terminated. 
NCSBE  shall  not  be  liable  to  Contractor  for  damages  of  any  kind  (general,  special  or  consequential)  as  a 
result  of  such  termination. 

6)  Contract  Transfer.  NCSU-TOPS  shall  not  transfer  any  interest  in  this  agreement  without  prior  written 
agreement  from  NCSBE. 

7)  Contract  Personnel.  NCSU-TOPS  agrees  that  it  has,  or  will  secure  at  its  own  expense,  all  personnel  required 
to  perform  the  services  set  forth  in  this  agreement.  NCSU-TOPS  shall  not  hire,  employ  or  contract  with  any 
person  currently  employed  by  NCSBE  to  furnish  any  material  or  perform  any  service  on  this  contract  without 
first  assuring  compliance  with  applicable  laws  of  the  State  of  North  Carolina  and  regulations  of  the  State 
Personnel  Commission  relating  to  dual  or  secondary  employment  by  State  employees.  In  the  event  NCSU- 
TOPS  hires,  employs  or  contracts  with  any  employee  of  the  NCSBE,  that  fact  will  be  immediately  reported  in 
writing,  referencing  the  contract  number,  to  the  Purchasing  Section,  Department  of  Public  Instruction. 

8)  Key  Personnel.  NCSU-TOPS  shall  not  substitute  for  key  personnel  assigned  to  the  performance  of  this 
contract  without  prior  written  approval  from  the  NCSBE  Project  Coordinator.  Key  personnel  are  defined  as 
those  individuals  named,  either  by  title  or  by  individual  name,  in  NCSU-TOPS's  offer. 

9)  Subcontractor  Approval.  None  of  the  work  to  be  performed  under  this  contract  by  NCSU-TOPS  or  its 
employees  shall  be  subcontracted  without  the  prior  written  approval  of  NCSBE  Project  Coordinator,  unless 
such  arrangement  was  specified  in  its  offer. 

1 0)  Contract  Audit.  NCSU-TOPS  agrees  that  NCSBE  has  the  right  to  audit  the  records  of  NCSU-TOPS 
pertaining  to  this  contract  both  during  the  period  of  this  contract  and  for  three  years  after  completion  hereof. 
Contractor  agrees  to  maintain  all  pertinent  documents  and  records  relating  to  this  contract  for  said  period  of 
time  following  completion  of  the  contract  period. 

1 1)  Contract  Modifications.  This  contract  may  be  amended  only  by  written  amendments  duly  executed  by  and 
between  NCSBE  and  NCSU-TOPS.  However,  to  take  advantage  of  unforeseen  opportunities  the  NCSBE 
Project  Coordinator  may  make  minor  modifications  that  (a)  do  not  change  the  intent  of  the  contract  or  the 
scope  of  NCSU-TOPS's  performance;  (b)  do  not  increase  NCSU-TOPS's  total  compensation  or  method  of 
payment;  and  (c)  either  improve  the  overall  quality  of  the  product  or  service  to  the  State  without  increasing 
the  cost,  or  reduce  the  total  cost  of  the  product  or  service  without  reducing  the  quantity  or  quality.  All  such 
minor  modifications  to  the  contract  must  be  recorded  in  writing  and  signed  by  NCSU-TOPS,  Project 
Coordinator  and  placed  on  file  with  the  Purchasing  Section,  NCSBE. 

12)  Tennination  for  Cause.  If,  through  any  cause,  NCSU-TOPS  fails  to  fulfill  in  a  timely  and  proper  manner  the 
obligations  under  this  contract,  NCSBE  will  thereupon  have  the  right  to  tenninate  this  contract  by  giving 
written  notice  to  NCSU-TOPS  of  such  termination  and  by  specifying  the  effective  date  thereof,  at  least  ten 
(10)  days  before  the  effective  date  of  such  termination.  In  such  event,  NCSU-TOPS  will  receive  just  and 
equitable  compensation  for  any  satisfactorily  completed  work.  Notwithstanding,  NCSU-TOPS  shall  not  be 
relieved  of  liability  to  NCSBE  for  damages  sustained  by  NCSBE  by  virtue  of  any  breach  of  this  contract  or 
failure  to  perform,  and  NCSBE  may  withhold  payment  to  NCSU-TOPS  for  the  purpose  of  set-off  until  such 
time  as  the  exact  amount  of  damages  due  NCSBE  from  such  breach  or  failure  can  be  determined. 
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1 3)  Contractor  Bankruptcy.  The  filing  of  a  petition  of  bankruptcy  or  insolvency  by  or  against  NCSU-TOPS  will 
terminate  this  contract. 

14)  Ownership  of  Contract  Products.  NCSU-TOPS  agrees  that  all  products,  records  and  data  tapes  produced 
under  this  contract  become  the  property  of  NCSBE. 

1 5)  Indemnification.  State  Agency  -  The  State  Agency's  liability  for  bodily  injury  and  property  damage  is 
established  by  the  North  Carolina  Tort  Claims  Act  (NC  General  Statute  Sec.  143-291  through  143-300.1), 
and  such  act  constitutes  the  limits  of  responsibility  of  NCSU-TOPS  under  this  agreement  concerning  personal 
and  bodily  injury  and  property  damage. 

16)  Contractor  Project  Coordinator.  NCSU-TOPS  must  designate  one  person  employed  at  the  executive  level, 
preferably  a  vice-president  or  the  president,  who  has  the  authority  to  expedite  work  or  make  adjustments  in 
scheduling.  This  person  will  be  contacted  by  NCSBE  Project  Coordinator  to  make  such  adjustments  only  in 
the  case  of  an  emergency. 

17)  Severability.  No  condition  in  this  document  shall  be  construed  to  limit  the  enforceability  of  any  other 
condition  herein.  In  the  event  any  term  or  condition  contained  in,  or  made  a  part  of  this  document  by 
reference,  is  negated  or  vacated  for  any  reason  whatsoever  all  other  provisions  shall  remain  fully  valid  and 
enforceable. 

1 8)  Payment  Schedule:  NCSU-TOPS  shall  submit  to  the  NCDPI  Project  Coordinator  an  invoice  requesting 
payments  under  tins  contract  on  a  monthly  schedule.  NCSU-TOPS  must  submit  a  final  invoice  to  the  NCDPI 
Project  Coordinator  on  or  before  the  contract  ending  date  for  payment  of  all  contract  products  and  services  to 
the  NCDPI  that  are  to  be  rendered  under  this  Agreement. 
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